A method for comparing multiple imputation techniques: A case study on the U.S. national COVID cohort collaborative.

Healthcare datasets obtained from Electronic Health Records have proven to be extremely useful for assessing associations between patients' predictors and outcomes of interest. However, these datasets often suffer from missing values in a high proportion of cases, whose removal may introduce severe bias. Several multiple imputation algorithms have been proposed to attempt to recover the missing information under an assumed missingness mechanism. Each algorithm presents strengths and weaknesses, and there is currently no consensus on which multiple imputation algorithm works best in a given scenario. Furthermore, the selection of each algorithm's parameters and data-related modeling choices are also both crucial and challenging. In this paper we propose a novel framework to numerically evaluate strategies for handling missing data in the context of statistical analysis, with a particular focus on multiple imputation techniques. We demonstrate the feasibility of our approach on a large cohort of type-2 diabetes patients provided by the National COVID Cohort Collaborative (N3C) Enclave, where we explored the influence of various patient characteristics on outcomes related to COVID-19. Our analysis included classic multiple imputation techniques as well as simple complete-case Inverse Probability Weighted models. Extensive experiments show that our approach can effectively highlight the most promising and performant missing-data handling strategy for our case study. Moreover, our methodology allowed a better understanding of the behavior of the different models and of how it changed as we modified their parameters. Our method is general and can be applied to different research fields and on datasets containing heterogeneous types.

Generalisable long COVID subtypes: findings from the NIH N3C and RECOVER programmes.

BACKGROUND: Stratification of patients with post-acute sequelae of SARS-CoV-2 infection (PASC, or long COVID) would allow precision clinical management strategies. However, long COVID is incompletely understood and characterised by a wide range of manifestations that are difficult to analyse computationally. Additionally, the generalisability of machine learning classification of COVID-19 clinical outcomes has rarely been tested. METHODS: We present a method for computationally modelling PASC phenotype data based on electronic healthcare records (EHRs) and for assessing pairwise phenotypic similarity between patients using semantic similarity. Our approach defines a nonlinear similarity function that maps from a feature space of phenotypic abnormalities to a matrix of pairwise patient similarity that can be clustered using unsupervised machine learning. FINDINGS: We found six clusters of PASC patients, each with distinct profiles of phenotypic abnormalities, including clusters with distinct pulmonary, neuropsychiatric, and cardiovascular abnormalities, and a cluster associated with broad, severe manifestations and increased mortality. There was significant association of cluster membership with a range of pre-existing conditions and measures of severity during acute COVID-19. We assigned new patients from other healthcare centres to clusters by maximum semantic similarity to the original patients, and showed that the clusters were generalisable across different hospital systems. The increased mortality rate originally identified in one cluster was consistently observed in patients assigned to that cluster in other hospital systems. INTERPRETATION: Semantic phenotypic clustering provides a foundation for assigning patients to stratified subgroups for natural history or therapy studies on PASC. FUNDING: NIH (TR002306/OT2HL161847-01/OD011883/HG010860), U.S.D.O.E. (DE-AC02-05CH11231), Donald A. Roux Family Fund at Jackson Laboratory, Marsico Family at CU Anschutz.

Metformin is associated with reduced COVID-19 severity in patients with prediabetes.

AIMS: Studies suggest that metformin is associated with reduced COVID-19 severity in individuals with diabetes compared to other antihyperglycemics. We assessed if metformin is associated with reduced incidence of severe COVID-19 for patients with prediabetes or polycystic ovary syndrome (PCOS), common diseases that increase the risk of severe COVID-19. METHODS: This observational, retrospective study utilized EHR data from 52 hospitals for COVID-19 patients with PCOS or prediabetes treated with metformin or levothyroxine/ondansetron (controls). After balancing via inverse probability score weighting, associations with COVID-19 severity were assessed by logistic regression. RESULTS: In the prediabetes cohort, when compared to levothyroxine, metformin was associated with a significantly lower incidence of COVID-19 with "mild-ED" or worse (OR [95% CI]: 0.636, [0.455-0.888]) and "moderate" or worse severity (0.493 [0.339-0.718]). Compared to ondansetron, metformin was associated with lower incidence of "mild-ED" or worse severity (0.039 [0.026-0.057]), "moderate" or worse (0.045 [0.03-0.069]), "severe" or worse (0.183 [0.077-0.431]), and "mortality/hospice" (0.223 [0.071-0.694]). For PCOS, metformin showed no significant differences in severity compared to levothyroxine, but was associated with a significantly lower incidence of "mild-ED" or worse (0.101 [0.061-0.166]), and "moderate" or worse (0.094 [0.049-0.18]) COVID-19 outcome compared to ondansetron. CONCLUSIONS: Metformin use is associated with less severe COVID-19 in patients with prediabetes or PCOS.

NSAID use and clinical outcomes in COVID-19 patients: a 38-center retrospective cohort study.

BACKGROUND: Non-steroidal anti-inflammatory drugs (NSAIDs) are commonly used to reduce pain, fever, and inflammation but have been associated with complications in community-acquired pneumonia. Observations shortly after the start of the COVID-19 pandemic in 2020 suggested that ibuprofen was associated with an increased risk of adverse events in COVID-19 patients, but subsequent observational studies failed to demonstrate increased risk and in one case showed reduced risk associated with NSAID use. METHODS: A 38-center retrospective cohort study was performed that leveraged the harmonized, high-granularity electronic health record data of the National COVID Cohort Collaborative. A propensity-matched cohort of 19,746 COVID-19 inpatients was constructed by matching cases (treated with NSAIDs at the time of admission) and 19,746 controls (not treated) from 857,061 patients with COVID-19 available for analysis. The primary outcome of interest was COVID-19 severity in hospitalized patients, which was classified as: moderate, severe, or mortality/hospice. Secondary outcomes were acute kidney injury (AKI), extracorporeal membrane oxygenation (ECMO), invasive ventilation, and all-cause mortality at any time following COVID-19 diagnosis. RESULTS: Logistic regression showed that NSAID use was not associated with increased COVID-19 severity (OR: 0.57 95% CI: 0.53-0.61). Analysis of secondary outcomes using logistic regression showed that NSAID use was not associated with increased risk of all-cause mortality (OR 0.51 95% CI: 0.47-0.56), invasive ventilation (OR: 0.59 95% CI: 0.55-0.64), AKI (OR: 0.67 95% CI: 0.63-0.72), or ECMO (OR: 0.51 95% CI: 0.36-0.7). In contrast, the odds ratios indicate reduced risk of these outcomes, but our quantitative bias analysis showed E-values of between 1.9 and 3.3 for these associations, indicating that comparatively weak or moderate confounder associations could explain away the observed associations. CONCLUSIONS: Study interpretation is limited by the observational design. Recording of NSAID use may have been incomplete. Our study demonstrates that NSAID use is not associated with increased COVID-19 severity, all-cause mortality, invasive ventilation, AKI, or ECMO in COVID-19 inpatients. A conservative interpretation in light of the quantitative bias analysis is that there is no evidence that NSAID use is associated with risk of increased severity or the other measured outcomes. Our results confirm and extend analogous findings in previous observational studies using a large cohort of patients drawn from 38 centers in a nationally representative multicenter database.

Abdominal Computed Tomography Imaging Findings in Hospitalized COVID-19 Patients: A Year-Long Experience and Associations Revealed by Explainable Artificial Intelligence.

The aim of this retrospective study is to assess any association between abdominal CT findings and the radiological stage of COVID-19 pneumonia, pulmonary embolism and patient outcomes. We included 158 adult hospitalized COVID-19 patients between 1 March 2020 and 1 March 2021 who underwent 206 abdominal CTs. Two radiologists reviewed all CT images. Pathological findings were classified as acute or not. A subset of patients with inflammatory pathology in ACE2 organs (bowel, biliary tract, pancreas, urinary system) was identified. The radiological stage of COVID pneumonia, pulmonary embolism, overall days of hospitalization, ICU admission and outcome were registered. Univariate statistical analysis coupled with explainable artificial intelligence (AI) techniques were used to discover associations between variables. The most frequent acute findings were bowel abnormalities (n = 58), abdominal fluid (n = 42), hematomas (n = 28) and acute urologic conditions (n = 8). According to univariate statistical analysis, pneumonia stage > 2 was significantly associated with increased frequency of hematomas, active bleeding and fluid-filled colon. The presence of at least one hepatobiliary finding was associated with all the COVID-19 stages > 0. Free abdominal fluid, acute pathologies in ACE2 organs and fluid-filled colon were associated with ICU admission; free fluid also presented poor patient outcomes. Hematomas and active bleeding with at least a progressive stage of COVID pneumonia. The explainable AI techniques find no strong relationship between variables.

Variations in volume of emergency surgeries and emergency department access at a third level hospital in Milan, Lombardy, during the COVID-19 outbreak.

BACKGROUND: During the recent outbreak of COVID-19 (coronavirus disease 2019), Lombardy was the most affected region in Italy, with 87,000 patients and 15,876 deaths up to May 26, 2020. Since February 22, 2020, well before the Government declared a state of emergency, there was a huge reduction in the number of emergency surgeries performed at hospitals in Lombardy. A general decrease in attendance at emergency departments (EDs) was also observed. The aim of our study is to report the experience of the ED of a third-level hospital in downtown Milan, Lombardy, and provide possible explanations for the observed phenomena. METHODS: This retrospective, observational study assessed the volume of emergency surgeries and attendance at an ED during the course of the pandemic, i.e. immediately before, during and after a progressive community lockdown in response to the COVID-19 pandemic. These data were compared with data from the same time periods in 2019. The results are presented as means, standard error (SE), and 95% studentized confidence intervals (CI). The Wilcoxon rank signed test at a 0.05 significance level was used to assess differences in per-day ED access distributions. RESULTS: Compared to 2019, a significant overall drop in emergency surgeries (60%, p < 0.002) and in ED admittance (66%, p ≅ 0) was observed in 2020. In particular, there were significant decreases in medical (40%), surgical (74%), specialist (ophthalmology, otolaryngology, traumatology, and urology) (92%), and psychiatric (60%) cases. ED admittance due to domestic violence (59%) and individuals who left the ED without being seen (76%) also decreased. Conversely, the number of deaths increased by 196%. CONCLUSIONS: During the COVID-19 outbreak the volume of urgent surgeries and patients accessing our ED dropped. Currently, it is not known if mortality of people who did not seek care increased during the pandemic. Further studies are needed to understand if such reductions during the COVID-19 pandemic will result in a rebound of patients left untreated or in unwanted consequences for population health.

Artificial Intelligence in Predicting Clinical Outcome in COVID-19 Patients from Clinical, Biochemical and a Qualitative Chest X-Ray Scoring System

Purpose: To determine the performance of a chest radiograph (CXR) severity scoring system combined with clinical and laboratory data in predicting the outcome of COVID-19 patients. Materials and Methods: We retrospectively enrolled 301 patients who had reverse transcriptase-polymerase chain reaction (RT-PCR) positive results for COVID-19. CXRs, clinical and laboratory data were collected. A CXR severity scoring system based on a qualitative evaluation by two expert thoracic radiologists was defined. Based on the clinical outcome, the patients were divided into two classes: moderate/mild (patients who did not die or were not intubated) and severe (patients who were intubated and/or died). ROC curve analysis was applied to identify the cut-off point maximizing the Youden index in the prediction of the outcome. Clinical and laboratory data were analyzed through Boruta and Random Forest classifiers. Results: The agreement between the two radiologist scores was substantial (kappa = 0.76). A radiological score ≥ 9 predicted a severe class: sensitivity = 0.67, specificity = 0.58, accuracy = 0.61, PPV = 0.40, NPV = 0.81, F1 score = 0.50, AUC = 0.65. Such performance was improved to sensitivity = 0.80, specificity = 0.86, accuracy = 0.84, PPV = 0.73, NPV = 0.90, F1 score = 0.76, AUC= 0.82, combining two clinical variables (oxygen saturation [SpO2]), the ratio of arterial oxygen partial pressure to fractional inspired oxygen [P/F ratio] and three laboratory test results (C-reactive protein, lymphocytes [%], hemoglobin). Conclusion: Our CXR severity score assigned by the two radiologists, who read the CXRs combined with some specific clinical data and laboratory results, has the potential role in predicting the outcome of COVID-19 patients.

Volume-of-Interest Aware Deep Neural Networks for Rapid Chest CT-Based COVID-19 Patient Risk Assessment.

Since December 2019, the world has been devastated by the Coronavirus Disease 2019 (COVID-19) pandemic. Emergency Departments have been experiencing situations of urgency where clinical experts, without long experience and mature means in the fight against COVID-19, have to rapidly decide the most proper patient treatment. In this context, we introduce an artificially intelligent tool for effective and efficient Computed Tomography (CT)-based risk assessment to improve treatment and patient care. In this paper, we introduce a data-driven approach built on top of volume-of-interest aware deep neural networks for automatic COVID-19 patient risk assessment (discharged, hospitalized, intensive care unit) based on lung infection quantization through segmentation and, subsequently, CT classification. We tackle the high and varying dimensionality of the CT input by detecting and analyzing only a sub-volume of the CT, the Volume-of-Interest (VoI). Differently from recent strategies that consider infected CT slices without requiring any spatial coherency between them, or use the whole lung volume by applying abrupt and lossy volume down-sampling, we assess only the "most infected volume" composed of slices at its original spatial resolution. To achieve the above, we create, present and publish a new labeled and annotated CT dataset with 626 CT samples from COVID-19 patients. The comparison against such strategies proves the effectiveness of our VoI-based approach. We achieve remarkable performance on patient risk assessment evaluated on balanced data by reaching 88.88%, 89.77%, 94.73% and 88.88% accuracy, sensitivity, specificity and F1-score, respectively.

Explainable Machine Learning for Early Assessment of COVID-19 Risk Prediction in Emergency Departments.

Between January and October of 2020, the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus has infected more than 34 million persons in a worldwide pandemic leading to over one million deaths worldwide (data from the Johns Hopkins University). Since the virus begun to spread, emergency departments were busy with COVID-19 patients for whom a quick decision regarding in- or outpatient care was required. The virus can cause characteristic abnormalities in chest radiographs (CXR), but, due to the low sensitivity of CXR, additional variables and criteria are needed to accurately predict risk. Here, we describe a computerized system primarily aimed at extracting the most relevant radiological, clinical, and laboratory variables for improving patient risk prediction, and secondarily at presenting an explainable machine learning system, which may provide simple decision criteria to be used by clinicians as a support for assessing patient risk. To achieve robust and reliable variable selection, Boruta and Random Forest (RF) are combined in a 10-fold cross-validation scheme to produce a variable importance estimate not biased by the presence of surrogates. The most important variables are then selected to train a RF classifier, whose rules may be extracted, simplified, and pruned to finally build an associative tree, particularly appealing for its simplicity. Results show that the radiological score automatically computed through a neural network is highly correlated with the score computed by radiologists, and that laboratory variables, together with the number of comorbidities, aid risk prediction. The prediction performance of our approach was compared to that that of generalized linear models and shown to be effective and robust. The proposed machine learning-based computational system can be easily deployed and used in emergency departments for rapid and accurate risk prediction in COVID-19 patients.

Chest CT in patients with a moderate or high pretest probability of COVID-19 and negative swab.

OBJECTIVES: We aimed to assess the diagnostic performance of CT in patients with a negative first RT-PCR testing and to identify typical features of COVID-19 pneumonia that can guide diagnosis in this case. METHODS: Patients suspected of COVID-19 with a negative first RT-PCR testing were retrospectively revalued after undergoing CT. CT was reviewed by two radiologists and classified as suspected COVID-19 pneumonia, non-COVID-19 pneumonia or negative. The performance of both first RT-PCR result and CT was evaluated by using sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV) and area under the curve (AUC) and by using the second RT-PCR test as the reference standard. CT findings for confirmed COVID-19 positive or negative were compared by using the Pearson chi-squared test (P values < 0.05) RESULTS: Totally, 337 patients suspected of COVID-19 underwent CT and nasopharyngeal swabs in March 2020. Eighty-seven out of 337 patients had a negative first RT-PCR result; of these, 68 repeated RT-PCR testing and were included in the study. The first RT-PCR test showed SE 0, SP = 100%, PPV = NaN, NPV = 70%, AUC = 50%, and CT showed SE = 70% SP = 79%, PPV = 86%, NPV = 76%, AUC = 75%. The most relevant CT variables were ground glass opacity more than 50% and peripheral and/or perihilar distribution. DISCUSSION: Negative RT-PCR test but positive CT features should be highly suggestive of COVID-19 in a cluster or community transmission scenarios, and the second RT-PCR test should be promptly requested to confirm the final diagnosis.